Determining when to drive autonomously

ABSTRACT

Aspects of the disclosure relate generally to determining whether an autonomous vehicle should be driven in an autonomous or semiautonomous mode (where steering, acceleration, and braking are controlled by the vehicle&#39;s computer). For example, a computer may maneuver a vehicle in an autonomous or a semiautonomous mode. The computer may continuously receive data from one or more sensors. This data may be processed to identify objects and the characteristics of the objects. The detected objects and their respective characteristics may be compared to a traffic pattern model and detailed map information. If the characteristics of the objects deviate from the traffic pattern model or detailed map information by more than some acceptable deviation threshold value, the computer may generate an alert to inform the driver of the need to take control of the vehicle or the computer may maneuver the vehicle in order to avoid any problems.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.13/444,215, filed on Apr. 11, 2012, now U.S. Pat. No. 8,718,861, theentire disclosure of which are hereby incorporated herein by reference.

BACKGROUND

Autonomous vehicles use various computing systems to aid in thetransport of drivers from one location to another. Some autonomousvehicles may require some initial input or continuous input from anoperator, such as a pilot or driver. Other systems, for exampleautopilot systems, may be used only when the system has been engaged,which permits the operator to switch from a manual mode (where theoperator exercises a high degree of control over the movement of thevehicle) to an autonomous mode (where the vehicle essentially drivesitself) to modes that lie somewhere in between.

These autonomous vehicles may maneuver themselves between locationsusing highly detailed maps in conjunction with sensors for detecting thevehicle's surroundings. If the detailed maps are incorrect, it may beparticularly difficult for the vehicle to navigate without input fromthe driver.

In some driving situations, a driver may not feel particularly saferelying completely upon the vehicle to maneuver itself. For example, adriver may feel less safe in areas with dense traffic or in closeproximity to a vehicle that is moving erratically. Thus, some driversmay feel the need to continuously monitor the vehicle's location in casethe driver must take control of the vehicle from the vehicle'sautonomous computing system. This may lessen the usefulness of anautonomous vehicle and a driver's sense of safety.

BRIEF SUMMARY

One aspect of the disclosure provides a method. The method includesreceiving data from one or more sensors associated with a vehicle;detecting an object and a characteristic for the detected object basedon the received data; determining, by a processor, a deviation value forthe detected object based on a comparison of the characteristic for thedetected object to the traffic pattern model information, the trafficpattern model information including an expected range of values for acharacteristic of objects in the road; comparing the deviation value toa threshold deviation value for an expected range of values for thecharacteristic; and when the deviation value is outside of the thresholddeviation value, providing a notification to a driver of the vehicle.

In one example, the method also includes receiving input from the driverindicating that the driver has taken control of the vehicle. In anotherexample, the characteristic includes a position of the detected objectand the deviation value is determined by calculating a differencebetween the position of the detected object and an expected range ofvalues for position defined in the traffic pattern model information. Inyet another example, the characteristic includes a speed of the detectedobject and the deviation value is determined by calculating a differencebetween the speed of the detected object and an expected range of valuesfor speed defined in the traffic pattern model information. In a furtherexample, the characteristic includes a trajectory of the detected objectand the deviation value is determined by calculating a differencebetween the trajectory of the detected object and an expected range ofvalues for trajectory defined in the traffic pattern model information.In still a further example, the method also includes detecting a secondobject and a second characteristic for the second detected object basedon the received data; determining a second deviation value for thesecond detected object based on a comparison of the secondcharacteristic for the second detected object to the traffic patternmodel information; comparing the second deviation value to the thresholddeviation value for an expected range of values for the secondcharacteristic; and when the deviation value is within the thresholddeviation value and the second deviation value is outside of the secondthreshold deviation value, providing the notification to the driver thevehicle. In another example, the method also includes detecting a secondobject and a second characteristic for the detected object based on thereceived data; determining a second deviation value for the seconddetected object based on a comparison of the second characteristic forthe second detected object to the traffic pattern model information; andbefore providing the notification, determining whether the seconddeviation value is within the second threshold deviation value based ona comparison of the second deviation value to the second thresholddeviation value. In yet another example, the method also includesdetermining a second deviation value for the detected object based on acomparison of the characteristic for the detected object to detailed mapinformation describing expected features of the road and characteristicsof the expected features; comparing the second deviation value to asecond threshold deviation value for the expected characteristics of theexpected features; and when the second deviation value is outside of thesecond threshold deviation value, providing the notification to thedriver of the vehicle.

Another aspect of the disclosure provides a method. The method includesreceiving data from one or more sensors associated with a vehicle;detecting an object and a characteristic for the detected object basedon the received data; determining, by a processor, a deviation value forthe detected object based on a comparison of the characteristic todetailed map information describing expected features of the road andcharacteristics of the expected features; comparing the deviation valueto a threshold deviation value for the expected characteristics of theexpected features; when the deviation value is outside of the thresholddeviation value, providing a notification to the driver of the vehicle.

In one example, the method also includes receiving input from the driverindicating that the driver has taken control of the vehicle. In anotherexample, the characteristic includes a position of the detected objectand the deviation value is determined by calculating a differencebetween the position of the detected object and an expectedcharacteristic for position defined in the detailed map information. Inyet another example, the characteristic includes a shape of the detectedobject and the deviation value is determined by calculating a differencebetween the shape of the detected object and an expected characteristicfor shape defined in the detailed map information. In a further example,the characteristic includes a size of the detected object and thedeviation value is determined by calculating a difference between thesize of the detected object and an expected characteristic for sizedefined in the detailed map information. In still a further example, themethod also includes detecting a second object and a secondcharacteristic for the second detected object based on the receiveddata; determining a second deviation value for the second detectedobject based on a comparison of the second characteristic and the mapinformation; comparing the second deviation value to a second thresholddeviation value for the expected characteristics of the expectedfeatures; and when the deviation value is within the threshold deviationvalue and the second deviation value is outside of the second thresholddeviation value, providing a notification to the driver of the vehicle.In another example, the method also includes detecting a second objectand a second characteristic for the second detected object based on thereceived data; determining a second deviation value for the seconddetected object based on a comparison of the second characteristic andthe detailed map information; and before providing the notification,determining whether the second deviation value is within the secondthreshold deviation value.

A further aspect of the disclosure provides a method. The methodincludes receiving data from one or more sensors associated with avehicle; detecting an object and a characteristic for the detectedobject based on the received data; determining, by a processor, adeviation value for the detected object based on a comparison of thecharacteristic and traffic pattern model information, the trafficpattern model information including an expected range of values for acharacteristic of objects in the road; comparing the deviation value toa threshold deviation value for the expected range of values for thecharacteristic of the given object; and when the deviation value isoutside of the threshold deviation value, maneuvering, without inputfrom a driver, the vehicle defensively.

In one example, maneuvering the vehicle defensively includes slowing thevehicle down, changing lanes, or increasing the distance between thevehicle and another object. In another example, the characteristicincludes a position of the detected object and the deviation value isdetermined by calculating a difference between the position of thedetected object and an expected range of values for position defined inthe traffic pattern model information. In yet another example, thecharacteristic includes a speed of the detected object and the deviationvalue is determined by calculating a difference between the speed of thedetected object and an expected range of values for speed defined in thetraffic pattern model information. In a further example, thecharacteristic includes a trajectory of the detected object and thedeviation value is determined by calculating a difference between thetrajectory of the detected object and an expected range of values fortrajectory defined in the traffic pattern model information. In still afurther example, the method also includes detecting a second object anda second characteristic for the detected object based on the receiveddata; determining a second deviation value for the second detectedobject based on a comparison of the second characteristic to the trafficpattern model information; comparing the second deviation value to asecond threshold deviation value for the expected range of values forthe characteristic of the given object; and when the deviation value iswithin the threshold deviation value and the second deviation value isoutside of the second threshold deviation value, maneuvering the vehicledefensively. In another example, detecting a second object and a secondcharacteristic for the detected object based on the received data;determining a second deviation value for the second detected objectbased on a comparison of the second characteristic and the trafficpattern model information; and before maneuvering the vehicledefensively, determining whether the second deviation value is withinthe second threshold deviation value based on a comparison of the seconddeviation value to the second threshold deviation value. In yet anotherexample, the method also includes determining a second deviation valuefor the detected object based on a comparison of the characteristic forthe detected object and detailed map information describing expectedfeatures of the road and characteristics of the expected features;comparing the second deviation value to a second threshold deviationvalue for the expected characteristics of the expected features; whenthe second deviation value is outside of the second threshold deviationvalue, maneuvering the vehicle defensively.

A further aspect of the disclosure provides a method. The methodincludes receiving data from one or more sensors associated with avehicle; detecting an object and a characteristic for the detectedobject based on the received data; determining, by a processor, adeviation value for the detected object based on a comparison of thecharacteristic for the detected object and detailed map informationdescribing expected features of the road and characteristics of theexpected features; comparing the second deviation value to a secondthreshold deviation value for the expected characteristics of theexpected features; and identifying the mismatched area when the seconddeviation value is outside of the second threshold deviation value.

In one example, the characteristic includes a position of the detectedobject and the deviation value is determined by calculating a differencebetween the position of the detected object and an expectedcharacteristic for position defined in the detailed map information. Inanother example, the characteristic includes a shape of the detectedobject and the deviation value is determined by calculating a differencebetween the shape of the detected object and an expected characteristicfor shape defined in the detailed map information. In yet anotherexample, the characteristic includes a size of the detected object andthe deviation value is determined by calculating a difference betweenthe size of the detected object and an expected characteristic for sizedefined in the detailed map information. In a further example, themethod also includes detecting a second object and a secondcharacteristic for the detected object based on the received data;determining a second deviation value for the second detected objectbased on a comparison of the second characteristic and the detailed mapinformation; comparing the second deviation value to a second thresholddeviation value for the expected characteristics of the expectedfeatures; when the deviation value is within the threshold deviationvalue and the second deviation value is outside of the second thresholddeviation value, identifying a mismatched area; and maneuvering, withoutinput from the driver, the vehicle to avoid the mismatched area. Instill a further example, the method also includes detecting a secondobject and a second characteristic for the detected object based on thereceived data; determining a second deviation value for the seconddetected object based on a comparison of the second characteristic andthe detailed map information; and before identifying the mismatchedarea, determining whether the second deviation value is within thesecond threshold deviation value.

Still a further aspect of the disclosure provides a device. The deviceincludes memory storing traffic pattern model information including anexpected range of values for a characteristic of objects in the road.The device also includes a processor coupled to the memory. Theprocessor is configured to receive data from one or more sensorsassociated with a vehicle; detect an object and a characteristic for thedetected object based on the received data; determine a deviation valuefor the detected object based on a comparison of the characteristic forthe detected object to the traffic pattern model information; comparethe deviation value to a threshold deviation value for the expectedrange of values for the characteristic of the given object; and when thedeviation value is outside of the threshold deviation value, provide anotification to a driver.

In one example, the processor is also configured to slow the vehicledown if the driver does not take control after the notification isprovided. In another example, the processor is also configured tomaneuver the vehicle into a different lane if the driver does not takecontrol after the notification is provided.

Another aspect of the disclosure provides a device. The device includesmemory storing detailed map information describing expected features ofthe road and characteristics of the expected features. The device alsoincludes a processor coupled to the memory. The processor is configuredto receive data from one or more sensors associated with a vehicle;detect an object and a characteristic for the detected object based onthe received data; determine a deviation value for the detected objectbased on a comparison of the characteristic and the detailed mapinformation; compare the second deviation value to a threshold deviationvalue for the expected characteristics of the expected features; andwhen the deviation value is outside of the threshold deviation value,provide a notification to a driver of the vehicle.

In one example, the processor is also configured to slow the vehicledown if the driver does not take control after the notification isprovided. In another example, the processor is also configured to whenthe deviation value is outside of the threshold deviation value,identify a mismatched area and to maneuver the vehicle to avoid themismatched area if the driver does not take control after thenotification is provided.

Yet another aspect of the disclosure provides a tangiblecomputer-readable storage medium on which computer readable instructionsof a program are stored. The instructions, when executed by a processor,cause the processor to perform a method. The method includes receivingdata from one or more sensors associated with a vehicle; detecting anobject and a characteristic for the detected object based on thereceived data; determining a deviation value for the detected objectbased on a comparison of the characteristic and traffic pattern modelinformation, the traffic pattern model information including an expectedrange of values for a characteristic of objects in the road; comparingthe deviation value to a threshold deviation value for the expectedrange of values for the characteristic of the given object; and when thedeviation value is outside of the threshold deviation value,maneuvering, without input from a driver, the vehicle defensively.

A further aspect of the disclosure provides a tangible computer-readablestorage medium on which computer readable instructions of a program arestored. The instructions, when executed by a processor, cause theprocessor to perform a method. The method includes receiving data fromone or more sensors associated with a vehicle; detecting an object and acharacteristic for the detected object based on the received data;determining a deviation value for the detected object based on acomparison of the characteristic for the detected object and detailedmap information describing expected features of the road andcharacteristics of the expected features; comparing the second deviationvalue to a second threshold deviation value for the expectedcharacteristics of the expected features; and identifying the mismatchedarea when the second deviation value is outside of the second thresholddeviation value.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an example autonomous vehicle system in accordance withaspects of the disclosure.

FIG. 2 is a diagram of example detailed map information in accordancewith aspects of the disclosure.

FIG. 3 is a diagram of an example interior of an autonomous vehicle.

FIG. 4 is an example diagram of a vehicle in accordance with aspects ofthe disclosure.

FIG. 5 is an example of a road in accordance with aspects of thedisclosure.

FIG. 6 is example data in accordance with aspects of the disclosure.

FIG. 7 is a flow diagram in accordance with aspects of the disclosure.

FIG. 8 is an example of a road in accordance with aspects of thedisclosure.

FIG. 9 is example data in accordance with aspects of the disclosure.

FIG. 10 is another flow diagram in accordance with aspects of thedisclosure.

FIG. 11 is a further flow diagram in accordance with aspects of thedisclosure.

DETAILED DESCRIPTION

As shown in FIG. 1, an autonomous vehicle system 100 in accordance withone aspect of the disclosure includes a vehicle 101 with variouscomponents. While certain aspects of the disclosure are particularlyuseful in connection with specific types of vehicles, the vehicle 101may be any type of vehicle including, but not limited to, cars, trucks,motorcycles, busses, boats, airplanes, helicopters, lawnmowers,recreational vehicles, amusement park vehicles, farm equipment,construction equipment, trams, golf carts, trains, and trolleys. Thevehicle may have one or more computers, such as a computer 110containing a processor 120, memory 130 and other components typicallypresent in general purpose computers.

The processor 120 may be any conventional processor, such ascommercially available CPUs. Alternatively, the processor may be adedicated device such as an ASIC or other hardware-based processor.Although FIG. 1 functionally illustrates the processor, memory, andother elements of computer 110 as being within the same block, it willbe understood by those of ordinary skill in the art that the processor,computer, or memory may actually comprise multiple processors,computers, or memories that may or may not be stored within the samephysical housing. For example, memory may be a hard drive or otherstorage media located in a housing different from that of computer 110.Accordingly, references to a processor or computer will be understood toinclude references to a collection of processors or computers ormemories that may or may not operate in parallel. Rather than using asingle processor to perform the steps described herein, some of thecomponents, such as steering components, acceleration and decelerationcomponents, may each have their own processor that only performscalculations related to the component's specific function.

In various aspects described herein, the processor 120 may be locatedremotely from the vehicle 101 and communicate with the vehicle 101wirelessly. In other aspects, some of the processes described herein areexecuted on a processor disposed within the vehicle 101 and others by aremote processor, including taking the steps necessary to execute asingle maneuver.

The memory 130 stores information accessible by the processor 120,including instructions 132 and data 134 that may be executed orotherwise used by the processor 120. The memory 130 may be of any typecapable of storing information accessible by the processor, including acomputer-readable medium, or other medium that stores data that may beread with the aid of an electronic device, such as a hard-drive, memorycard, ROM, RAM, DVD or other optical disks, as well as otherwrite-capable and read-only memories. Systems and methods may includedifferent combinations of the foregoing, whereby different portions ofthe instructions and data are stored on different types of media.

The instructions 132 may be any set of instructions to be executeddirectly (such as machine code) or indirectly (such as scripts) by theprocessor. For example, the instructions may be stored as computer codeon the computer-readable medium. In that regard, the terms“instructions” and “programs” may be used interchangeably herein. Theinstructions may be stored in object code format for direct processingby the processor, or in any other computer language including scripts orcollections of independent source code modules that are interpreted ondemand or compiled in advance. Functions, methods, and routines of theinstructions are explained in more detail below.

The data 134 may be retrieved, stored or modified by processor 120 inaccordance with the instructions 132. For instance, although the claimedsubject matter is not limited by any particular data structure, the datamay be stored in computer registers, in a relational database as a tablehaving a plurality of different fields and records, XML documents orflat files. The data may also be formatted in any computer-readableformat. By further way of example only, image data may be stored asbitmaps comprised of grids of pixels that are stored in accordance withformats that are compressed or uncompressed, lossless (e.g., BMP) orlossy (e.g., JPEG), and bitmap or vector-based (e.g., SVG), as well ascomputer instructions for drawing graphics. The data may comprise anyinformation sufficient to identify the relevant information, such asnumbers, descriptive text, proprietary codes, references to data storedin other areas of the same memory or different memories (including othernetwork locations) or information that is used by a function tocalculate the relevant data.

The data may include environmental data that was obtained at a previouspoint in time and is expected to persist regardless of the vehicle'spresence in the environment. For example, data 134 may include detailedmap information 136, e.g., highly detailed maps detecting the shape andelevation of roadways, lane lines, intersections, crosswalks, speedlimits, traffic signals, buildings, signs, real time trafficinformation, or other such features and information. These features maybe persistent, for example, as described in more detail below, when thevehicle 101 approaches the location of a feature in the detailed mapinformation, the computer 110 may expect to detect the feature. Thedetailed map information may also include explicit speed limitinformation associated with various roadway segments. The speed limitdata may be entered manually or scanned from previously taken images ofa speed limit sign using, for example, optical-character recognition.The detailed map information may also include two-dimensionalstreet-level imagery, such as highly detailed image data depicting thesurroundings of a vehicle from the vehicle's point-of-view. The detailedmap information may also include three-dimensional terrain mapsincorporating one or more of the objects listed above.

The detailed map information 136 may also include zone information,indicating zones that are unsuitable for driving autonomously. Forexample, an on-ramp, off-ramp, or other complicated or high trafficareas may be identified as such zones as a driver may feel the need tocontinuously monitor the vehicle in case the driver must take control.Other zones may be identified as unsuitable for any driving, such as asidewalk, river, mountain, cliff, desert, etc.

Again, although the detailed map information is depicted herein as animage-based map, the map information need not be entirely image based(for example, raster). For example, the detailed map information mayinclude one or more roadgraphs or graph networks of information such asroads, lanes, intersections, and the connections between these features.Each feature may be stored as graph data and may be associated withinformation such as a geographic location and whether or not it islinked to other related features, for example, a stop sign may be linkedto a road and an intersection, etc. In some examples, the associateddata may include grid-based indices of a roadgraph to allow forefficient lookup of certain roadgraph features.

FIG. 2 depicts a pictorial representation of detailed map information136 corresponding to the features of a road 210, for example, thosefeatures which computer 110 may expect to detect while on road 210.Markers 220 and 228 represent boundaries on each of the sides of theroad 210. Each boundary may be a curb, guardrail, highway divider, orother form of barrier. In some examples, the detailed map information136 may identify each type of boundary differently. In the example ofFIG. 2, for instance, the boundaries represented by markers 220 and 228are both curbs.

Markers 222, 224, and 226 each represent lane lines painted on thesurface of the road 210. Each lane line may be a double solid yellowline, single solid white line, single dashed white line, or other formof lane line. In some examples, the detailed map information 136 mayidentify each type of lane line differently. In the example of FIG. 2,for instance, the lane lines represented by markers 222 and 226 aresingle dashed white lines, indicating that vehicles may cross these lanelines in order to change lanes. The lane line represented by marker 224is a double solid yellow line, indicating that vehicles may not crossthis lane line in order to change lanes.

The detailed map information 136 may contain additional informationregarding characteristics of the stored objects. For instance, thedetailed map information may include information regarding the height ofthe curbs 220 and 228 (e.g., elevating 4 centimeters above the surfaceof the road 210). The detailed map information may include informationregarding the width of each of the lane lines 222-226. Informationregarding the width of lane lines 222 and 226, which are single dashedwhite lines, may indicate that each line has a width of 8 centimeters.Information regarding the width of lane line 224, which is a doublesolid yellow line, may indicate that this line has a width of 20centimeters.

Shaded regions 213 and 215 each represent zones unsuitable for driving.In some examples, the detailed map information 136 may identify eachtype of zone differently (i.e., detecting a desert differently from ariver). In the example of FIG. 2, for instance, the off-road regionsrepresented by shaded areas 212 and 214 are a desert.

Data 134 may also include traffic pattern model information 138, e.g., ahighly detailed model indicating the distribution of typical or expectedspeeds, trajectories, locations, accelerations/decelerations (changes inspeed), or other such characteristics of vehicles or other movingobjects on the locations of the detailed map information. This data maybe generated, for example, by observing how vehicles, pedestrians,bicycles, etc. move at different locations in the detailed mapinformation.

In some aspects, the traffic pattern model information may indicateinformation pertinent to the entire road. For example, the trafficpattern model information may indicate a range of speeds for vehiclestravelling along the road 210 of FIG. 2, specific to each particularlane or independent of the lanes. The traffic pattern model information138 may indicate a range of trajectories for vehicles travelling inparticular lanes. For example, referring to FIG. 2, the range oftrajectories for lane 218 may range from due north (driving straight inlane 218) to 10 degrees west of north (changing into lane 216).Similarly, the traffic pattern model information 138 may indicate adistribution of trajectories for vehicles travelling in lane 216 rangingfrom due north (driving straight in lane 216) to 10 degrees east ofnorth (changing into lane 218).

Data 134 may also include one or more threshold deviation values 139.The threshold values may be set manually by an administrator based onthe traffic pattern model information and the detailed map informationin order to promote safe driving of the vehicle by computer. Thesethreshold deviation values may indicate acceptable differences from thetraffic pattern model information 138. For example, a thresholddeviation value 139 may be set at 20 MPH lower than the slowest speedindicated by the traffic pattern model information 138. This would meanthat if the traffic pattern model information 138 for the road 210indicates that vehicles on the road 210 travel at speeds ranging between50 MPH and 70 MPH, a threshold deviation value 139 for road 210 may beset at 30 MPH, 20 MPH lower than the slowest speed (50 MPH). In anotherexample, a threshold deviation value 139 may be set 10 MPH higher thanthe fastest speed indicated by the traffic pattern model information138. This would mean that given the same range of speeds as in theexample above, a threshold deviation value may be set at 80 MPH, 10 MPHhigher than the fastest speed (70 MPH).

In some aspects, one or more threshold deviation values 139 maycorrespond to characteristics of the features of the detailed mapinformation 136. Thus, the threshold deviation values may indicateacceptable differences from the detailed map information. For example, athreshold deviation value 139 for the width of lane lines in thedetailed map information may be set at 6 inches. Thus, if the detailedmap information indicates that lane line 224 in the center of the road210 is 18 inches wide, then a first threshold deviation value 139 forthe lane line 224 may be set at 12 inches, 6 inches thinner than thelane line 224, and a second threshold deviation value 139 may be set at26 centimeters, 6 inches wider than the lane line 224.

The threshold deviation values 139 may be stored in the data 134 withthe traffic pattern model information 138 and/or detailed mapinformation 136. Alternatively, the threshold deviation values 139 maybe stored separately from the traffic pattern model information and thedetailed map information 136.

In one example, computer 110 may be incorporated into vehicle 101. Thevehicle 101 may also include one or more user interfaces for allowingcommunication between a driver of the vehicle 101 and the computer 110.The user interfaces may include status indicators, electronic displays,and user input devices built into the interior of the vehicle 101.

FIG. 3 depicts a design of the interior of an autonomous vehicle. Theautonomous vehicle 101 may include all of the features of anon-autonomous vehicle, for example: a steering apparatus, such assteering wheel 310; a navigation display apparatus, such as navigationdisplay 315; and a gear selector apparatus, such as gear shifter 320.The vehicle may also have various user input devices, such as gearshifter 320, touch screen 317, or button inputs 319, for activating ordeactivating one or more autonomous driving modes and for enabling apassenger or driver 390 to provide information, such as a navigationdestination, to the computer 110.

Vehicle 101 may include one or more additional displays. For example,the vehicle may include a display 325 for displaying informationregarding the status of the autonomous vehicle or its computer. Inanother example, the vehicle may include a status bar 330, to indicatethe current status of vehicle 101. In the example of FIG. 3, status bar330 displays “D” and “2 mph” indicating that the vehicle is presently indrive mode and is moving at 2 miles per hour (MPH). In that regard, thevehicle may display text on an electronic display, illuminate portionsof vehicle 101, such as steering wheel 310, or provide various othertypes of indications.

Returning to FIG. 1, the computer 110 may be capable of communicatingwith various components of the vehicle 101. For example, the computer110 may be in communication with the vehicle's central processor 160 andmay send and receive information from the various systems of vehicle101, for example the braking 180, acceleration 182, signaling 184, andnavigation 186 systems in order to control the movement, speed, etc. ofvehicle 101. In addition, when engaged, computer 110 may control some orall of these functions of vehicle 101 and thus be fully or merelypartially autonomous. It will be understood that although varioussystems and computer 110 are shown within vehicle 101, these elementsmay be external to vehicle 101 or physically separated by largedistances.

The vehicle 101 may also have one or more components for detecting thestatus of the vehicle. For example, the vehicle 101 may include ageographic position component 150 in communication with the computer 110for determining the geographic location of the device. The positioncomponent may include a GPS receiver to determine the device's latitude,longitude and/or altitude position. Other location systems such aslaser-based localization systems, inertial-aided GPS, or camera-basedlocalization may also be used to identify the location of the vehicle.The location of the vehicle may include an absolute geographicallocation, such as latitude, longitude, and altitude as well as relativelocation information, such as location relative to other carsimmediately around it which can often be determined with less noise thanabsolute geographical location. The location of the vehicle may alsoindicate whether the vehicle 101 is underground (e.g., detecting thatthe vehicle 101 is in a tunnel or a cave) or above ground.

The vehicle 101 may also include an accelerometer, gyroscope or otherdirection/speed detection device 152 to determine the direction andspeed of the vehicle or changes thereto. By way of example only, thedirection/speed detection device 152 may determine its pitch, yaw orroll (or changes thereto) relative to the direction of gravity or aplane perpendicular thereto. The direction/speed detection device 152may also track increases or decreases in speed and the direction of suchchanges. The device's provision of location and orientation data as setforth herein may be provided automatically to the user, computer 110,other computers and combinations of the foregoing.

The vehicle may also be equipped with one or more sensors 144 fordetecting objects external to the vehicle such as other vehicles,obstacles in the roadway, traffic signals, signs, trees, etc. Thesensors 144 may include lasers, sonar, radar, cameras or any otherdetection devices which record data which may be processed by thecomputer 110.

The sensors 144 may be mounted on the vehicle 101 to collect informationregarding the environment of the vehicle 101. As shown in FIG. 4, thevehicle 101 may include lasers 410 and 411, mounted on the front and topof the vehicle, respectively. In one example, laser 410 may have a rangeof 150 meters, a thirty degree vertical field of view, and a thirtydegree horizontal field of view. In another example, laser 411 may havea range of 50-80 meters, a thirty degree vertical field of view, and a360 degree horizontal field of view. Various other ranges andconfigurations of one or more lasers may also be used. The lasers410/411 may provide the vehicle 101 with range and intensity informationwhich the computer may use to detect the location and distance ofvarious objects that absorb/reflect energy from the lasers 410/411. Inone aspect, the lasers 410/411 may measure the distance between thevehicle 101 and the object surfaces facing the vehicle by spinning onits axis and changing its pitch.

The vehicle 101 may also include various radar units, such as those usedfor adaptive cruise control systems. The radar units may be located onthe front and back of the vehicle 101 as well as on either side of thefront bumper. As shown in the example of FIG. 4, vehicle 101 includesradar units 420-423 located on the side (only one side being shown),front and rear of the vehicle. In one example, each of these radar unitsmay have a range of 200 meters for an 18 degree field of view as well asa range of 60 meters for a 56 degree field of view. Various other rangesand configurations of one or more radar units may also be used.

A variety of cameras may also be mounted on the vehicle 101. The camerasmay be mounted at predetermined distances so that the parallax from theimages of 2 or more cameras may be used to compute the distance tovarious objects. As shown in FIG. 4, the vehicle 101 may include 2cameras 430-431 mounted under a windshield 340 near the rear view mirror(not shown). In one example, camera 430 may include a range of 200meters and a 30 degree horizontal field of view, while camera 431 mayinclude a range of 100 meters and a 60 degree horizontal field of view.Various other ranges and configurations of one or more cameras may alsobe used.

The vehicle 101 may also include a transmitter and receiver 425. Thetransmitter/receiver 425 may receive and transmit information wirelesslyaccording to various communications protocols, such as cellular (e.g.3G, 4G) or WiFi (e.g. 802.11, 8021b, g, n, or other such standards). Thetransmitter/receiver 425 may also allow for inter-vehicle communication.The transmitter/receiver may also communicate with roadside sensors,such as a camera or laser stationed on the side of a road. As shown inFIG. 4, the transmitter/receiver 425 may be connected to a servercomputer 455 (having a processor, memory, and instructions, not shown)via a wireless network 445. The server computer 455 may storeinformation used by the computer 110 when controlling the vehicle 101.Such information may include maps, information about traffic patterns,road conditions, and so forth. The server computer 455 may receive fromvehicle 101 (via transmitter/receiver 425) map updates, map corrections,traffic pattern updates, traffic pattern corrections, as well as otherinformation. The server computer 455 may store the received informationin memory and/or transmit the information among other autonomousvehicles on the road.

The aforementioned sensors 144 may allow the vehicle 101 to evaluate andpotentially respond to its environment in order to maximize safety forthe driver, other drivers, as well as objects or people in theenvironment. It will be understood that the vehicle types, number andtype of sensors, the sensor locations, the sensor fields of view, andthe sensors' sensor fields are merely exemplary. Various otherconfigurations may also be utilized.

In addition to the sensors 144 described above, the computer 110 mayalso use input from sensors typical of non-autonomous vehicles. Forexample, these sensors may include tire pressure sensors, enginetemperature sensors, brake heat sensors, brake pad status sensors, tiretread sensors, fuel sensors, oil level and quality sensors, air qualitysensors (for detecting temperature, humidity, or particulates in theair), etc.

Many of these sensors provide data that is processed by the computer inreal-time, that is, the sensors 144 may continuously update their outputto reflect the environment being sensed at or over a range of time, andcontinuously or as-demanded provide that updated output to the computerso that the computer can determine whether the vehicle's then-currentdirection or speed should be modified in response to the sensedenvironment.

Operations in accordance with aspects of the disclosure will now bedescribed with reference to the figures. It should be understood thatthe following operations do not have to be performed in the preciseorder described below. Rather, various operations can be handled in adifferent order or simultaneously. It should also be understood thatthese operations do not have to be performed all at once. For instance,some operations may be performed separately from other operations.

The computer 110 of autonomous vehicle system 100 may maneuver thevehicle 101 autonomously or semiautonomously. For example, the computer110 may receive information from the sensors 144 and positioningcomponents 150, 152. This received information may be used to identifythe location of the vehicle 101. The location may be used to identifyrelevant portions of the detailed map information 136, traffic patternmodel information 138, etc. In some examples, the vehicle may use thedetailed map information 136 to refine its location estimate, forexample, by comparing the location of objects detected from the sensordata to the detailed map information. Using the sensor data as well asthe stored detailed map information 136, the computer 110 may alsocontrol the movement, speed, etc. of vehicle 101. This may includetaking actions such as activating a brake for braking 180, anaccelerator for acceleration 182, or controlling the steering withoutcontinued input from a driver.

In the example of FIG. 5, the computer 110 maneuvers the vehicle 101along the road 210 discussed above with regard to the detailed mapinformation of FIG. 2. In this example, lanes 512 and 514 of the road210 are designated for southbound traffic, as indicated by directionarrows 532 and 534 (not necessarily present on the road). Lanes 516 and518 are designated for northbound traffic, as indicated by directionarrows 536 and 538 (also not necessarily present on the road). Lanelines 522, 524, and 526 indicate the boundaries between lanes 512-518while curbs 520 and 528 establish boundaries for each side of the road210. In addition to the features of road 210 described with regard toFIG. 2 above, in this example, FIG. 5 includes several other vehiclesdriving on road 210. For example, vehicle 542 is driving south in lane512, vehicle 544 is driving south in lane 514, vehicle 546 is drivingnorth mostly in lane 516 (but partially in lane 518), and vehicle 548 isdriving south (against the flow of traffic) in lane 518.

The computer 110 may process the data received from the sensors todetect objects and/or features of the road. The sensor data may becontinuously received by the computer 110, such that the computer maydetect the presence of an object at different times. In addition todetecting the presence of an object, the vehicle may determine a set ofcharacteristics. As the data is taken at different times, thesecharacteristics may include, for example, the locations, speeds,trajectories, types of object, etc. In some examples, the computer 110may also compare the characteristics and/or sensor data to the detailedmap information 136 to increase the accuracy of these determinations.

For example, FIG. 6 is a pictorial representation of the objectsdetected by the computer 110 from the sensor data as compared to the mapinformation from FIG. 2 for roadway 210. For ease of understanding, FIG.6 depicts the detected objects 620-648 overlaid with detailed mapinformation for the road 210. In this example, objects 622, 624, and626, represent the computer's detection of the presence of lane lines522, 524, and 526 (show in FIG. 5), while objects 620 and 628 representthe computer's detection of the presence of curbs 520 and 528 (shown inFIG. 5). For example, the curb 520 may be detected by the sensors. Thecomputer may process the sensor data and identify object 620. In thisexample, object 620 is represented by a bounding box approximating thelocation and shape of curb 520 based on the sensor information. Similarprocessing may be conducted for each of the objects 522, 524, 526, and528. Using a similar analysis, the computer 110 may also identifyobjects 642, 644, 646, and 648 representing vehicles 542, 544, 546, and548 (shown only in FIG. 5). In this example, the location of objects620, 622, 624, 626, and 628 generally line up with the location ofobjects 220, 222, 224, 226, and 228 of the map information.

If the sensor data is collected over a period of time, the computer 110may also determine that objects 620, 622, 624, 626, and 628 arestationary, while objects 642, 644, 646, and 648 are moving. In additionto the location of these objects, the computer 110 may also determineother characteristics such as the speed, trajectory, and possible type(vehicle, pedestrian, bicycle, etc.) of these objects from the sensordata. As noted above, the computer 110 may also compare thecharacteristics and/or sensor data to the detailed map information 134to increase the accuracy of these determinations. For example, given thesize, speed, direction, and location of 642, 644, 646, and 648, thecomputer 110 may determine that these objects are likely to be othervehicles.

The computer may compare the characteristics of the detected objects totraffic pattern model information 138 in order to determine a deviationvalue. One or more characteristics in the traffic pattern modelinformation 138 for the relevant road may be compared to one or morecharacteristics of a detected object. Based on this comparison, thecomputer 110 may calculate a deviation value representing the deviationbetween one or more characteristics of the detected object and one ormore characteristics of the traffic pattern model information.

In one aspect, a characteristic of the traffic pattern model informationfor a particular section of road may indicate that vehicles (or simplymoving objects) on this particular section of road 210 typically drivewithin a range of speeds. This may be compared to the speed of detectedobjects which are likely to be vehicles (or simply moving objects) inthe road. Using the examples of road 210 and FIG. 6, a characteristic ofthe traffic pattern model information 138 for road 210 may indicate thatvehicles (or simply moving objects) typically move between 60 and 70MPH. The computer 110 may determine or calculate a deviation value (herea difference in speeds) between the speed of each of the objects likelyto be vehicles (here, 642, 644, 646, and 648) and the range of speeds inthe traffic pattern model information (50-70 MPH). In an example,computer 110 may determine that objects 646 and 648 are driving atapproximately 15 and 20 MPH, respectively. In this example, thedeviation values for each of objects 646 and 648 may be 35 and 30 MPH,respectively, or −35 and −30 MPH.

In another aspect, a characteristic of the traffic pattern modelinformation for a particular section of road may indicate how vehiclestypically drive on the road, for example, between curbs or between lanelines heading in the direction of the lanes. In other words, a vehiclein a northbound lane should be driving north. This may be compared tothe location and trajectory of the detected objects which are likely tobe vehicles or simply moving objects in the road. Returning to theexample of road 210 and the features of FIG. 6, a characteristic of thetraffic pattern model information 138 may indicate traffic pattern modelinformation that vehicles typically drive between lane lines in lanes.Specifically, vehicles located in the rightmost lane 218 of the road 210typically drive with a trajectory in a range between due north (i.e.,when driving straight in lane 216) and 10 degrees west of due north(i.e., when shifting from lane 218 to lane 216). The locations andtrajectories of each of the objects likely to be vehicles (here, 642,644, 646, and 648) may be compared to the typical locations andtrajectories of vehicles moving on road 210. For example, computer 110may determine that object 646 is not fully between lane lines 224 and226 or 226 and 228, with a trajectory of north. In this example, thedeviation value for the location of each of objects 646 may beapproximately 1 meter or 2 meters as object 646 may be approximately 1meter from the center of lane 216 or 2 meters from the center of lane218. In another example, object 648 is within lane lines 226 and 228,and moving with a trajectory of south. In this example, the range oftrajectories may be from 10 degrees west of north to 10 degrees east ofnorth, and the deviation value may be 170 degrees.

The deviation values may be compared to the threshold deviation values139 for the relevant characteristic or characteristics. For example, thecomputer 110 may determine whether a particular deviation value iswithin some relevant threshold deviation value or values from thetypical range of characteristics for similar objects on the same road.Thus, threshold deviation values for trajectories may be compared todeviation values for trajectories, threshold deviation values forlocations may be compared to deviation values for locations, and so on.In this regard, the computer 110 not only compares the detected objectsand characteristics to the traffic pattern model information 138, butalso determines whether these characteristics fall within an acceptabledeviation from the typical values. This may allow computer 110 todetermine the actual traffic conditions proximate to vehicle 101 in realtime.

Returning to the example of FIG. 6, objects 646 and 648 may be moving atapproximately 15 and 20 MPH. In this example, the computer 110 maycompare the determined deviation values, 35 and 30 MPH or −35 and −30MPH, respectively, to the threshold deviation values for speed ofvehicles on road 210. If the threshold deviation value is ±32 MPH, thenthe deviation value of object 646 is less than the threshold deviationvalue while the deviation value of object 648 is greater than thethreshold deviation value. This may indicate that object 646 is notwithin the acceptable range of speeds for road 210, while object 648 iswithin the acceptable range of speeds for road 210. If the deviationvalue is no more than 20 MPH below the typical range (50-70 MPH) and nogreater than 10 MPH above the typical range, the deviation values ofboth objects 646 and 648 are outside of this acceptable range. In theseexamples, both vehicles may be driving relatively slowly as compared tothe typical speeds for vehicles on road 210. This may be indicative ofheavy traffic, congestion, temporary roadwork, a broken down vehicle,etc.

When determining the relevant threshold deviation value, multiplecharacteristics of an object may be compared. For example, returning tothe example of FIG. 6, object 646 is not between lane lines 224 and 226or between lane lines 226 or 228, with a trajectory of north. In thisexample, the deviation value for object 646 may be approximately 1 meteror 2 meters as object 646 may be approximately 1 meter from the centerof lane 216 or 2 meters from the center of lane 218. Because thetrajectory of vehicle 646 is north, the deviation value for the locationof a vehicle within lane lines 224 and 226 or 226 and 228 may be only afew, for example 1-2, feet. Thus, object 646 may be outside of theacceptable ranges. If the trajectory of the vehicle 646 is at least 15degrees from north, this may indicate that vehicle 646 is changing lanesand thus the acceptable deviation value for location may be higher (forexample the width of an entire lane).

In another example, object 648 is within lane lines 226 and 228, andmoving with a trajectory of south. As object 646 is within lane lines226 and 228, the deviation value may be 170 degrees and the thresholddeviation value for vehicles traveling between lane lines 226 and 228may be 40 degrees from 10 degrees west of north or 10 degrees east ofnorth. Thus, in this example, object 648 may be outside of theacceptable range.

In addition to comparing the characteristics of a single detected objectto the traffic pattern information, the computer may processcharacteristics of multiple objects. For example, returning to theexample of objects 646 and 648 moving relatively slowly, the computer110 may look at the characteristics for both vehicles and compare themto threshold deviation values for multiple objects. In this example, therelatively slow speeds of both vehicles may indicate a traffic jam.

The above examples are examples of situations in which the computer 110may determine that a deviation value is outside of the relevantthreshold deviation value. The computer 110 may determine that adeviation value is outside of a threshold deviation value 139 in othersituations, such as those in which a driver of the vehicle 101 wouldfeel uncomfortable or feel the need to take control of the vehicle 101.

If the computer 110 determines that the deviation value does not exceedthe relevant threshold deviation value 139, the computer 110 maycontinue to maneuver the vehicle autonomously or semiautonomously.Alternatively, if the computer 110 determines that the deviation valueexceeds the relevant threshold deviation value, the computer 110 maygenerate and provide an alert to the driver. The alert may request thedriver take control of the vehicle 101. The alert may be an auralsignal, a visual signal, and/or any other signal that gets the attentionof the driver.

After alerting the driver, the autonomous vehicle system 100 thenreceives input from the driver. For example, the driver may take controlof the vehicle 101 by turning the steering wheel, applying the brake,applying the accelerator, pressing an emergency shut-off, etc. If thecomputer 110 does not receive input (or sufficient input) from thedriver, the computer 110 may navigate the vehicle 101 to the side of theroad (i.e., pull the vehicle over), reduce the speed of the vehicle 101,or bring the vehicle 101 to a complete stop.

In another example, rather than notifying the driver and requiring thedriver to take control of the vehicle, the computer 110 may take someother action. For example, computer 110 may take a defensive action,such as slowing down vehicle 101 or giving a larger distance betweenvehicle 101 and other objects. This may be especially useful underconditions such as very slow traffic where disengaging the autonomousmode may not actually be necessary. The computer may also maneuver thevehicle in order to change lanes to avoid whatever the problem may be.

In addition to taking all or some of the actions described above, thecomputer may also log the sensor data and traffic information for laterexamination. For example, this information may be transmitted to areporting server, such as server 455, for quality control review. Server455 may also send the information to other autonomous or non-autonomousvehicles. Alternatively, the computer 110 may transmit this informationdirectly to other vehicles. This may allow for a vehicle's computer todirectly identify traffic conditions, react to them, and share them withother vehicles without requiring action from the driver.

FIG. 7 illustrates an example flow chart 700 in accordance with some ofthe aspects described above. In block 702, the computer 110 maneuversthe vehicle. The computer 110 receives data from one or more sensors atblock 704. For example, as described above, the computer 110 may receiveinformation from a laser, camera, radar unit, etc. The computerprocesses the sensor data to detect one or more objects and one or morecharacteristics of those one or more objects at block 706. For eachdetected object, the computer 110 compares the one or morecharacteristics of the detected object to traffic pattern modelinformation at block 708. For example, a characteristic for speed of anobject may be compared to a typical speed of objects from the model. Thecomputer 110 then determines a deviation value for the comparedcharacteristics at block 710. The computer then determines whether thedeviation values are within threshold values for the relevantcharacteristics at block 712. If so, the computer 110 continues tomaneuver the vehicle at block 702, for example, based on the receivedsensor data. Returning to block 714, if the deviation values are notwithin the relevant threshold deviation values at block 712, thecomputer 110 generates an alert to the driver at block 714. For example,as described above, the computer 110 may cause a message to be displayedor played to the driver to inform him or her of the need to take controlof the vehicle.

In other aspects, once the computer 110 has detected the presence ofobjects and determined one or more characteristics for the objects, thecomputer 110 may compare the characteristics of the detected objects todetailed map information 136 in order to determine whether there is amismatch between the collected sensor data and the detailed mapinformation 136. This may be especially useful when the characteristicsof the road have changed and have not yet been updated in the detailedmap information. For example, FIG. 8 depicts a road 810 corresponding tothe same location as road 210. The lanes of the road 210 may berepositioned, for example, by repainting the lane lines of road 210 tochange the characteristics of the road. In this example, road 810 mayinclude four lanes, 812, 814, 816, and 818. Each of these lanes may beassociated with a respective traffic directions, direction arrows 832,834, 836 in the south bound direction and direction arrow 838 in thenorth bound direction. As with the examples of direction arrows 532,534, 536, and 538, these features, 832, 834, 836 and 838, are notnecessarily present on the road 810. The positions for the lane line 222and boundaries 220 and 228 on the road 210 (all shown in FIG. 2) areidentical to the positions of the lane line 822 and the boundaries 820and 828 on road 810 (shown in FIG. 8). The single lane line 226 and thedouble lane lines 224 of FIG. 2 have been removed from the surface ofthe road 810 in FIG. 8. In place of the removed lane lines, double lanelines 826 have been added to the surface of road 810 one meter to theright of the previous location of lane line 226, and a single lane line824 has been added to the surface of the new road 810 a half meter tothe right of the previous location of lane line 224.

As noted above, the computer 110 may detect objects from the sensor dataand the locations of the detected objects. The computer 110 may thencompare the detected objects and associated locations to the detailedmap information. As described above with regard to FIG. 2, the detailedmap information 136 for a particular section of road, such as road 210,may indicate the positions of stationary objects such as lane lines forthe particular section of road which the computer 110 may expect toobserve or detect from the sensor data.

FIG. 9 is a pictorial representation of the objects detected by thecomputer 110 from the sensor data while driving along road 810 or whatthe computer 110 identifies as road 210 based on the vehicle's location.In this example, detected objects 920, 922, 924, 926, and 928 areoverlaid with detailed map information for the road 210. In thisexample, while the data for the detected objects 920, 922, 924, 926, and928 may reflect the changes made to road 210, the detailed mapinformation 136 may not reflect these changes if the detailed mapinformation 136 has not been updated since the road 210 was changed.

Based on this comparison, the computer 110 may calculate a deviationvalue representing the difference from or deviation in position of thedetected objects and features of the detailed map information 136. Inthis example, the computer 110 may calculate a deviation value (here adifference in positions) between the position of each of the detectedobjects 920-928 and the positions of the objects 220-228 in the detailedmap information 136. For each of the detected objects 920-928, aseparate deviation value may be calculated between the object and eachof the objects 220-228 or between the position of the detected objects924 and 926 relative to features with similar shape, location, and/orsize characteristics. In this example, object 924 may be compared to theclosest feature of road 210, or double lane lines 224. Thus, thecharacteristics of object 924 may be compared with the characteristicsof double lane lines 224. If the object 924 is 2 feet from double lines224, the deviation value for the location of object 924 may be 2 feet.Similarly, object 926 may be compared to the closest feature of road210, or lane line 226. In this case, the deviation value for the object926 may be 3 feet from lane line 226.

In some examples, in addition to calculating a deviation value forposition, the computer 110 may calculate a deviation value (here adifference in shape) between the shape of each of the detected objects920-928 and the shapes of the objects 220-228 in the detailed mapinformation 136. For each of the detected objects 920-928, a separatedeviation value may be calculated between the characteristics of eachdetected object and each of the features of the map information withsimilar shape, location, and/or size characteristics. Returning to theexample of FIG. 9, detected object 924 may actually be a lane line witha detected width of approximately 6 inches, and object 926 may actuallybe a double lane line with a detected width of 18 inches. In thisexample, the deviation values for the detected object 924 may be zero(or negligibly small) when compared to the shape of objects 222 and 226(also having a width of 6 inches), and 12 inches when compared to theshape of object 224 (having a width of 18 inches, 12 inches greater thanthe detected width of object 924). Similarly, the deviation values forthe detected object 926 may be zero when compared to the shape of object224, but may be 12 inches when compared to the shape of objects 222 and226.

The deviation values may be compared to the threshold deviation values139 for the relevant characteristics of the feature or features of thedetailed map information. For example, the computer 110 may determinewhether a particular deviation value 139 is within an acceptablethreshold deviation value or values from the features of the detailedmap information. For example, the threshold deviation value for thelocation of lane lines may be a few inches or a foot whereas thethreshold deviation value for double lane lines may be more or less.Returning to the example of calculating deviation values for thepositions of detected objects 924 and 926, with deviation values of 2feet and 3 feet, respectively, both of these objects are not within thethreshold deviation values for double lane line 224 or lane lines 226.Thus, these detected objects may not be within the acceptable range oflocations for the double lane lines 224 and lane lines 226.

When determining the relevant threshold deviation value, the thresholddeviation value for one characteristic of the object may be affected bya deviation value for a second characteristic of the object. Forexample, returning to the example of FIG. 9, detected object 924 may bewithin the acceptable range of locations to be detected as lane line 224if the threshold deviation value 139 for the location of an object is 2feet. However, detected object 924 may not have the same size and shapecharacteristics as lane line 224. In this example, if the thresholddeviation value 139 for the shape of lane lines (such as lane line 226or double lane line 224) is 6 inches, then the deviation value for theshape of detected object 924, compared to double lane line 224, may begreater than or not within of the threshold deviation value 139.

In addition to comparing the characteristics of a single detected objectto the detailed map information, the computer may processcharacteristics of multiple objects. For example, returning to theexample of objects 924 and 926, the computer 110 may look at thecharacteristics for both lane lines and compare them to thresholddeviation values for multiple objects. In this example, the mismatchbetween the positions and shapes of these lane lines and the positionsand shapes of the objects in the detailed map information may indicate asignificant change in road 210.

The above examples are examples of situations in which the computer 110may determine that a deviation value is outside of the relevantthreshold deviation value. In other words, there is a mismatch betweenthe map information and the actual world. The computer 110 may determinethat a deviation value is outside of a threshold deviation value inother situations, such as those in which the computer 110 cannot detectobjects on a road that correspond with objects featured in the detailedmap information 136.

If the computer 110 determines that the deviation value is within therelevant threshold deviation value 139, the computer 110 may continue tomaneuver the vehicle 101 autonomously or semiautonomously.Alternatively, if the computer 110 determines that the deviation valueexceeds the relevant threshold deviation value 139, the computer 110 maygenerate and provide an alert to the driver. The alert may request thedriver take control of the vehicle 101. As noted above the alert may bean aural signal, a visual signal, and/or any other signal that gets theattention of the driver.

Again, after alerting the driver, the computer 110 may then receiveinput from the driver. For example, the driver may take control of thevehicle 101 by turning the steering wheel 310, applying the brake,applying the accelerator, pressing an emergency shut-off, etc. If thecomputer 110 does not receive input (or sufficient input) from thedriver, the computer 110 may navigate the vehicle 101 to the side of theroad (i.e., pull the vehicle over), reduce the speed of the vehicle 101,or bring the vehicle 101 to a complete stop.

As noted above, rather than notifying the driver and requiring thedriver to take control of the vehicle, the computer 110 may take someother action such as slow down vehicle 101 or avoid the mismatched areaby driving around it (if possible). In addition to taking all or some ofthe actions described above, the computer may also log the sensor dataand mismatch information for later examination. For example, thisinformation may be transmitted to a reporting server, such as server455, for quality control review. Server 455 may also send theinformation to other autonomous or non-autonomous vehicles.Alternatively, the computer 110 may transmit this information directlyto other vehicles. This may allow for a vehicle's computer to directlyidentify mismatch conditions, react to them, and share them with othervehicles without requiring action from the driver.

FIG. 10 illustrates an example flow chart 1000 in accordance with someof the aspects described above. In block 1002, the computer 110maneuvers the vehicle as described above. The computer 110 receives datafrom one or more sensors at block 1004. For example, as described above,the computer 110 may receive information from a laser, camera, radarunit, etc. The computer processes the sensor data to detect one or moreobjects and one or more characteristics of those one or more objects atblock 1006. For each detected object, the computer 110 compares the oneor more characteristics of the detected object to detailed mapinformation at block 1008. For example, a characteristic for thelocation or size of a stationary object may be compared to the locationsof objects in the detailed map information 136. The computer 110 thendetermines a deviation value for the compared one or morecharacteristics at block 1010. The computer determines whether thedeviation values are within threshold values for the relevantcharacteristics at block 1012. If so, the computer 110 continues tomaneuver the vehicle at block 1002, for example, based on the receivedsensor data. Returning to block 1014, if the deviation values are notwithin the relevant threshold deviation values at block 1012, thecomputer 110 generates an alert to the driver at block 1014. Forexample, as described above, the computer 110 may cause a message to bedisplayed or played to the driver to inform him or her of the need totake control of the vehicle.

The traffic pattern model information and detailed map informationdescribed above may also be used in combination to safely maneuver thevehicle 101. For example, FIG. 11 is a flow diagram 1100 illustratingone of many possible combinations of the aspects described above. Inblock 1102, the computer 110 maneuvers the vehicle as described above.The computer 110 receives data from one or more sensors at block 1104.For example, as described above, the computer 110 may receiveinformation from a laser, camera, radar unit, etc. The computerprocesses the sensor data to detect one or more objects and one or morecharacteristics of those one or more objects at block 1106. For eachdetected object, the computer 110 compares the one or morecharacteristics of the detected object to detailed map information atblock 1108. For example, the location, size, and/or shape of an objectmay be compared to the locations of objects in the detailed mapinformation 136. The computer 110 then determines a deviation value forthe compared one or more characteristics at block 1110. The computerthen determines whether the deviation values are within threshold valuesfor the relevant characteristics at block 1112. If the deviation valuesare not within the relevant threshold deviation values at block 1112,the computer 110 generates an alert to the driver at block 1120. Forexample, as described above, the computer 110 may cause a message to bedisplayed or played to the driver to inform him or her of the need totake control of the vehicle.

Returning to block 1112, if the deviation values are not within therelevant threshold deviation values, then for each detected object, thecomputer 110 compares the one or more characteristics of the detectedobject to traffic pattern model information at block 1114. For example,a characteristic for speed of an object may be compared to a typicalspeed of objects from the model. The computer 110 then determines adeviation value for the compared characteristics at block 1116. Thecomputer then determines whether the deviation values are withinthreshold values for the relevant characteristics at block 1118. If so,the computer 110 continues to maneuver the vehicle at block 1102, forexample, based on the received sensor data. Returning to block 1118, ifthe deviation values are within the relevant threshold deviation values,the computer 110 generates an alert to the driver at block 1120. Forexample, as described above, the computer 110 may cause a message to bedisplayed or played to the driver to inform him or her of the need totake control of the vehicle.

In addition to traffic pattern model information and the detailed mapinformation, the computer 110 may rely on other information to determinewhether to alert a driver to take control of the vehicle. For examplethe computer 110 may use various sensors such as atmospheric sensors ormoisture sensors to detect current weather conditions. This sensor datamay be supplemented by information received at the vehicle, for example,weather reports. If the weather conditions indicate an unsafe situationfor the vehicle, for example, icy conditions or heavy snow, the computer110 may generate and provide an alert to the driver as described above.

As these and other variations and combinations of the features discussedabove can be utilized without departing from the subject matter asdefined by the claims, the foregoing description of exemplaryimplementations should be taken by way of illustration rather than byway of limitation of the subject matter as defined by the claims. Itwill also be understood that the provision of the examples describedherein (as well as clauses phrased as “such as,” “e.g.”, “including” andthe like) should not be interpreted as limiting the claimed subjectmatter to the specific examples; rather, the examples are intended toillustrate only some of many possible aspects.

The invention claimed is:
 1. A method comprising: receiving, by one ormore processors, data collected by one or more sensors associated with avehicle; detecting, by the one or more processors, an object and acharacteristic of the detected object based on the received data;determining, by the one or more processors, a deviation value for thecharacteristic of the detected object based on a comparison of thecharacteristic of the detected object to a characteristic of an objectidentified in detailed map information; selecting a threshold deviationvalue based on the object identified in the detailed map information;and controlling, by the one or more processors, the vehicle based onwhether the deviation value satisfies the selected threshold deviationvalue.
 2. The method of claim 1, wherein selecting the thresholddeviation value is further based on a type of the object in the detailedmap information.
 3. The method of claim 1, wherein selecting thethreshold deviation value is further based on a type of thecharacteristic of the detected first object.
 4. The method of claim 1,further comprising: detecting a second object and a secondcharacteristic for the second detected object based on the receiveddata; determining a second deviation value for the second detectedobject based on a comparison of the second characteristic for the seconddetected object to a characteristic of a second object identified in thedetailed map information; selecting a second threshold deviation valuebased on the comparison detailed map information, wherein the selectedsecond threshold deviation value is different from the selectedthreshold deviation value; controlling the vehicle based on whether thesecond deviation value satisfies the selected second threshold deviationvalue.
 5. The method of claim 1, further comprising: detecting a secondobject and a second characteristic for the second detected object basedon the received data; determining a second deviation value for thesecond detected object based on a comparison of the secondcharacteristic for the second detected object to the characteristic ofthe object identified in the detailed map information; controlling thevehicle based on whether the second deviation value satisfies theselected threshold deviation value.
 6. The method of claim 1, whereinthe characteristic is a position of the detected object, and the methodfurther comprises: detecting a second characteristic for the detectedobject based on the received data, wherein the second characteristicindicates a shape of the detected object; determining a second deviationvalue for the second detected object based on a comparison of the secondcharacteristic for the detected object to a second characteristic of thesecond object identified in the detailed map information, wherein thesecond characteristic of the second object identified in the detailedmap information indicates an expected shape of the second object;selecting a second threshold deviation value based on the comparison ofthe second characteristic for the detected object to the secondcharacteristic of the second object identified in the detailed mapinformation, wherein the selected second threshold deviation value isdifferent from the selected threshold deviation value; controlling thevehicle based on whether the second deviation value satisfies theselected second threshold deviation value.
 7. The method of claim 1,further comprising providing a notification to a driver based on whetherthe deviation value satisfies the selected threshold deviation value. 8.A system, comprising one or more processors configured to: receive datacollected by one or more sensors associated with a vehicle; detect anobject and a characteristic of the detected object based on the receiveddata; determine a deviation value for the characteristic of the detectedobject based on a comparison of the characteristic of the detected firstobject to a characteristic of an object identified in detailed mapinformation; select a threshold deviation value based on the objectidentified in the detailed map information; and control the vehiclebased on whether the deviation value satisfies the selected thresholddeviation value.
 9. The system of claim 8, wherein selecting thethreshold deviation value is further based on a type of the object inthe detailed map information.
 10. The system of claim 8, whereinselecting the threshold deviation value is further based on a type ofthe characteristic of the detected first object.
 11. The system of claim8, wherein the one or more processors are further configured to: detecta second object and a second characteristic for the second detectedobject based on the received data; determine a second deviation valuefor the second detected object based on a comparison of the secondcharacteristic for the second detected object to a characteristic of asecond object identified in the detailed map information; select asecond threshold deviation value based on the comparison detailed mapinformation, wherein the selected second threshold deviation value isdifferent from the selected threshold deviation value; control thevehicle based on whether the second deviation value satisfies theselected second threshold deviation value.
 12. The system of claim 8,wherein the one or more processors are further configured to: detectinga second object and a second characteristic for the second detectedobject based on the received data; determine a second deviation valuefor the second detected object based on a comparison of the secondcharacteristic for the second detected object to the characteristic ofthe object identified in the detailed map information; control thevehicle based on whether the second deviation value satisfies theselected threshold deviation value.
 13. The system of claim 8, whereinthe characteristic is a position of the detected object, and the one ormore processors are further configured to: detect a secondcharacteristic for the detected object based on the received data,wherein the second characteristic indicates a shape of the detectedobject; determine a second deviation value for the second detectedobject based on a comparison of the second characteristic for thedetected object to a second characteristic of the second objectidentified in the detailed map information, wherein the secondcharacteristic of the second object identified in the detailed mapinformation indicates an expected shape of the second object; select asecond threshold deviation value based on the comparison of the secondcharacteristic for the detected object to the second characteristic ofthe second object identified in the detailed map information, whereinthe selected second threshold deviation value is different from theselected threshold deviation value; control the vehicle based on whetherthe second deviation value satisfies the selected second thresholddeviation value.
 14. The system of claim 8, wherein the one or moreprocessors are further configured to provide a notification to a driverbased on whether the deviation value satisfies the selected thresholddeviation value.
 15. The system of claim 8, further comprising thevehicle, and wherein the one or more processors are associated with thevehicle.
 16. A non-transitory, computer-readable storage medium on whichcomputer readable instructions of a program are stored, theinstructions, when executed by one or more processors, cause the one ormore processors to perform a method, the method comprising: receivingdata collected by one or more sensors associated with a vehicle;detecting an object and a characteristic of the detected object based onthe received data; determining a deviation value for the characteristicof the detected object based on a comparison of the characteristic ofthe detected first object to a characteristic of an object identified indetailed map information; selecting a threshold deviation value based onthe object identified in the detailed map information; and controllingthe vehicle based on whether the deviation value satisfies the selectedthreshold deviation value.
 17. The medium of claim 16, wherein selectingthe threshold deviation value is further based on a type of the objectin the detailed map information.
 18. The medium of claim 16, whereinselecting the threshold deviation value is further based on a type ofthe characteristic of the detected first object.
 19. The medium of claim16, wherein the method further comprises: detecting a second object anda second characteristic for the second detected object based on thereceived data; determining a second deviation value for the seconddetected object based on a comparison of the second characteristic forthe second detected object to a characteristic of a second objectidentified in the detailed map information; selecting a second thresholddeviation value based on the comparison detailed map information,wherein the selected second threshold deviation value is different fromthe selected threshold deviation value; controlling the vehicle based onwhether the second deviation value satisfies the selected secondthreshold deviation value.
 20. The medium of claim 16, wherein themethod further comprises: detecting a second object and a secondcharacteristic for the second detected object based on the receiveddata; determining a second deviation value for the second detectedobject based on a comparison of the second characteristic for the seconddetected object to the characteristic of the object identified in thedetailed map information; controlling the vehicle based on whether thesecond deviation value satisfies the selected threshold deviation value.